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Image; scanning the safe frame area to produce a digital 
image; extracting the machine readable metadata from 
the digital image; extracting the scene image from the 
digital image; and processing the scene image accord- 
ing to the extracted machine readable metadata. 
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Description 

[0001] The invehtron relates generally to the field of photography, and In particular to photofinish ing. More specifically, 
the invention relates to a method of photof inishing that employs an encoded data pattern placed on photographic film. 

5 [0002] The 35mm film systehn hks been on the market fpr many decades. This film system consists of light-sensitive, 
AgX media spooled into a cartridge of particular construction; cameras designed to handle this film^ and expose it to 
the image of a scene;. and photof inishing equipment designed to extract the film, develop It in a chemical bath, and 
print the image onto photo-sensitive paper. The elements of this film system have co-evolved over many decades into 
a coherent, working* system. Many vendors supply equipment to each.part.of this film system. 

10 [0003] Recently, one portion of the 35mm film system has changed. Photofinishing equipment that scans the film 
and converts the image to a digital fomi has been introduced. Digital data, in some cases, is used directly to create a 
viewable print, via digital writing. In other cases, the data is stored on a writable magnetic or optical disc, for use by 
the customer in a computer system. In still other cases, the data is sent by a networi< to a website or directly to the 
customer's computer. Even though this development is recent, many companies already supply this type of digitization 

IS equipment. The digitization equipment is widely dispersed. Some of the digitization equipment is placed at central 
locations to where the film cartridges are shipped, and other digital equipment is at retail locations. 
[0004] Skilled artisans know how to record data on photographic film, either magnetically in a magnette layer on the 
film, or optically as latent Images on the film. The data, generally known in the imapirig Industry as metadata, may 
contain Infomnatlon about the captured scene, or about the photographer's technical preferences, or even contain 

20 Information on how the image should be reproduced. l\/letadata is infonnnation associated with a picture or with a set 
of pictures, other than the actual image information Itself. When an image has been digitized, we refer to the infonmation 
that recreates the image as the pixel data; everything else is therefore metadata. 

[0005] One class of metadata records a photographer's request to modify the reproduction of a specific frame of film. 
For example, the photographer may request that the central region of the film frame, which is roughly one-quarter the 
25 area of the entire frame, be used to produce the final print (otherwise known as, pseudo zoom). Alternatively, in another 
implementation, the photographer may request that the Image be rendered as monochrome and have a sepia tint (I. 
e., a sepia feature). 

[0006] A second class of metadata adds information that does not modify the reproduced image. For example, the 
date on which each frame was exposed may be stored as metadata. 

30 [0007] In short, metadata may describe a scene; the device used to capture the scene; the intent of the photographer; 
the context of the picture; or a request for certain products to be created from the picture data. Metadata may also be 
used to improve the quality of reproductions; to classify pictures for easy retrieval; or for creating output products. 
[0008] Metadata is most useful when it is generated directly and automatically in the camera as the scene is captured 
or shortly thereafter. However, the metadata has to be transported by intermediate components. Afterwards, the meta- 

35 data is read and used by the processing equipment that creates prints or manages digital data. 

[0009] In the 35mm film system, the only practical mechanism for associating metadata with the Image or with a roll 
of images at the capture point is optically marking the film itself. These optical marks must be made in such a way that 
they can be differentiated from and not interfere with, useful scene recording. They must also be made In such a way 
that the equipment reading the film can easily measure and interpret them. 

40 [001 0] In the case of optically recorded metadata, the metadata can be read by photofinishing equipment to control 
photographic processing and printing operations. For example, US Patent 5,870,639 issued February 9, 1999 to Con- 
stable et al. discloses recording latent image metadata called "fat bits" on the marginal edges of an image frame and 
outside of the area reserved for the Image. The areas reserved for the image are herein refen-ed to as the safe frame 
areas of the film strip. The fat bits are later used to control the aspect ratio of a print produced from the image frame. 

45 [0011] It is also known to optically record data such as time and date by superimposing the time and/or date on the 
image within the safe frame area of the film. See, for example US Patent 5^51 9,463 issued May 21.1 996 to Nakamura 
et al. This data, however, Is not intended to be machine readable, and Is optically reproduced and viewable solely when 
a print of the image is made. 

[0012] There are a large variety of 35mm film cameras on the market. They range from one-time-use cameras that 
50 sell for less than ten dollars, to professional cameras that sell for thousands of dollars. Similarty, the features desired 
by users of these cameras vary greatly. Also, the allowable cost of a marking device for metadata In these cameras 
varies greatly. Nevertheless, the same photofinishing equipment is generally used for all 35mm color film, no matter 
what camera places the image on the film. 

[0013] In considering the introduction of cameras that write optical metadata to a 35mm film, the placement of these 
55 optical martcs on the film is critical . One choice Is to place the marks outside the safe frame areas, hence, either between 
or outside the film's perforations. A second choice is to place the mari<s In the safe frame area, but between the framed 
images. From the viewpoint of the camera designer, all of these methods have advantages and disadvantages, and 
all are quite practical. Both methods have the disadvantage of a limited available area and a limited data storage 



2 



BNSDOCIO: <EP 1213909A2_L> 



EP 1 213 909 A2 

capacity. 

[0014] Although digital scanning is becoming quite connmon, the processing equipnnent still requires special equip- 
nnent adaptations in order to handle any information recorded outside the safe frame area of the film. See tor example 
US Patent 5,665,950 issued Septjember 9, 1997 to Rottner et al. which discloses a bar code reader for reading "fat 
5 bits" recorded on the edge.s of photographic film. Because of the extra expense ©nd complexity, these special equipment 
adaptations for handling information recorded outside of the safe frame area of photographic film ai;e not likely to be 
widely deployed. ' 

[001 5] From the viewpoint of the photof Inlshing equipment designer, the aforementioned methods present very dif- 
ferent issues. As noied above, there are many types of digital photof Inishing equipment on the market. In every case, 

10 the ability to read this additional optteal metadata is an add-on to an existing design. In most cases, there is a need to 
retrofit ^equipment already in the field. Designing photoflnrshing equipment requires balancing mechanical, optical, 
electronic, and software tradeoffs. The introduction of additional information on the film, outside the safe frame area, 
poses substantial challenges in all these fields. In particular, the software design of photofinishing systems is quite 
complex, because there are many layers of software between the optical reading device and the image processing 

IS subsystem. 

[0016] Because metadata has proven so useful, there is substantial interest in increasing the amount of metadata 
embedded in film. However, heretofore substantial photofinishing hardware and software modifications were required 
to accommodate increased embedded metadata. 

[0017] There is a need, therefore, for an Improved method and apparatus for optically recording and recovering 
so metadata from photographic film that does not require additional and costly photofinishing hardware modifications. 
[001 8] The need is met according to the present Invention by providing a photofinishing method, including the steps 
of: exposing machine readable metadata and a scene image within an entire safe frame area on a fllm.strip; processing 
the filmstrip to produce a visible image Including the machine readable metadata and the scene image; scanning the 
safe frame area to produce a digital image; extracting the machine readable metadata from the digital image; extracting 
25 the scene image from the digital Image; and processing the scene image according to the extracted machine readable 
metadata. 

[0019] This invention has the advantage that the additional metadata is scanned and transported through the pho- 
tofinishing system easily, requiring entirely no modification in hardware and only minor modification In a small area of 
software. 

30 [0020] These and other aspects, objects, features and advantages of the present Invention will be more clearly 
understood and appreciated from a review of the following detailed description of the pretended embodiments and 
appended claims, and by reference to the accompanying drawings. 

Fig. 1 is a schematic diagram of a filmstrip having machine readable metadata according to the present invention; 
Fig. 2 is a schematic diagram Illustrating possible structures of encoded metadata used with the present Invention; 
Fig. 3 Is a schematic diagram of a photofinishing system according to the present invention; 
Fig. 4 Is a rear view of the mechanical configuration of a camera capable of generating "dot code," as shown in Fig. 2; 
Fig. 5 Is a side view of the mechanical configuration of a camera capable of generating "dot code," as shown in 
Fig. 2; 

Fig. 6 is a rear view of the mechanical configuration of a camera capable of generating a "one-dimensional bar 
code," as shown in Fig. 2; 

Fig. 7 is a side view of the mechanical configuration of a camera capable of generating "one-dimensional bar 
code," as shown in Fig. 2; 

Fig. 8 is a rearview of the mechanical configuration of acamera capable of generating "two-dimensional barcode," 
as shown in Fig. 2; 

Fig. 9 is a side view of the mechanical configuration of a camera capable of generating "two-dimensional bar code," 
as shown in Fig. 2; 

Fig. 10 is a flowchart for analyzing the presence of metadata; 
Fig. 11 Is a flowchart for searching for distinctive metadata; 
Fig. 12 Is a flowchart for forming a distinctive metadata ID; 
Fig. 13 is an example of a distinctive metadata pattern; and 

Fig. 14 is a further elaboration of fomning a distinctive metadata ID using the flowchart of Fig. 12. 

[0021] According to the present invention metadata is written as optical marks onto 35mm-fonnat photographic film 
in a camera, in a manner such that it can be read by all existing 35mm film scanners. This is accomplished by writing 
the optical marks in the region of the film commonly used for scene recording, generally known as the "safe frame 
area" of each frame on the film. These marks consist of a coded pattern of density produced by, e.g., LED devfces In 
the camera. Another part of this safe frame area In each frame will generally contain the image of the scene, cast by 
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a lens. The metadata pattern is designed to be well-isolated from the scene information, either spatially or by a distinctive 
• color pattern. Hence, the present invention affords a person skilled in the art the opportunity to place the metadata 
pattern within the central image area, thereby competing with the framed images, but also providing differentiation from 
the framed images in some manner. 

5 [0022] One cjass of metadata comprises a request by the photographer to modify the reproduction of a specific frame 
of film. For example, the photographer may request that a central region'of the film, roughly one-quarter the area of 
the entire frame, be used to produce the final print (i.e., pseudo zoom). Alternatively In another implementation, the 
photographer may request that the image be rendered in monochrome with a sepia tint (i.e., sepia feature). 
[0023] A second class of metadata adds infomiation that does not modify the reproduced image. For example, the 

10 date on which each frame was actually exposed may be stored as metadata. 

[0024] The developed density in the entire frame is read from the film by a scanner and passed serially through a 
number of hardware and software interfaces, and eventually delivered to a computational engine. This computational 
engine decodes the coded information and modifies the image data to produce the desired digital image. The coded 
infomiation may change' the way in which the image is reproduced, or may carry information about the scene or about 

IS the photographer's preferences. 

[0025] There is already a large deployed base of 35mm digital scanning and processing equipment. Even though 
new equipment is being developed, special adaptations for handling information recorded outside the nomiat image 
frame are unlikely to be widely deployed. This invention has the advantage that the additional metadata is scanned 
and transported through the photofinishing system easily and requires no modifbation In the hardware and only minor 

20 software modifications. 

[0026] According to the present invention, machine readable rrietadata is optically recorded on the part of the frame 
of film called the 'safe frame area.' The safe frame area, by industry convention, Is the minimal area on'the frame which 
is f^ead by every scanner for reproducing the image in the frame. The safe frame area also contains the captured image. 
The entire frame may be read by a scanner and sent to a computer, At this juncture, the computer is able to decode 

25 the metadata, thereby allowing a photofinisher to produce the desired digital image, for example, by applying digital 
resizing to the image. 

[0027] As shown in Fig. 1, a 35mhri film or filmstrip 10 (used Interchangeably herein) has a simple physical layout. 
A row of perforations 1 1 is found on each edge of the filmstrip 1 0. The region between these perforations 11 is commonly 
used to record the image of a scene 18. There isnt any natural framing defined on the filmstrip 10, therefore, conven- 
30 tionally, the sequential frames of images fall where they may on the filnnstrip 10. Many films have latent-image codes 
(not shown) written on them in the factory. These codes (a form of metadata) are used to identify the film manufacturer 
and film type. The codes are generally read by specialized devices In the photofinishing equipment. The codes are 
welhstandardlzed. The devices to read them add substantially to the complexity of designing the photofinishing equip- 
ment. 

35 [0028] Referring to Figs. 1 , 4 and 5, according to one embodiment of the invention, a conventional 35mm film or 
filmstrip 10, delivered from the factory without any special optical markings, includes safe frame areas 12 that are 
normally intended for receiving exposures. The film 10 is employed in a camera 14 (shown in Fig. 4) that is designed 
as described below for use with the present invention. The camera 14, is used to capture an image and record it on 
filmstrip 10. According to the preseht invention, the safe frame areas 12 that normally receive an optical exposure are 

40 subdivided into two regions. One region 16 of the safe frame area 12 receives an optical exposure (not shown) from 
a lens 102 (shown in Fig. 5) of the camera 14. A second region 19 receives an encoded metadata exposure 20 (further 
shown in Fig, 2), from a modulated light source such as a light emitting diode (LED) array 101, recessed in a portion 
of the film gate 104 (shown in Fig. 4) In the camera 14. The camera 14 is more fully described below according to Figs. 
4, 5, 6, 7, 8 and 9. 

45 [0029] Referring to Fig. 2 and elements 20a-20c, the structure of the encoded metadata 20 can take various forms 
depending on the sophistication of the camera 14 and the needs of the application. For example. In a very simple 
camera, a stationery set of LED's can write a pattern of density dots (described as a "dot-code" 20a). The camera 14 
has a mechanical configuration that generates the dot code 20a and Is shown in Fig. 4. 

[0030] Fig. 4 shows a rear view of the camera 14 against which the filmstrip 10 lies upon, and a side view of the 
50 camera 14. A lens 102 Is placed roughly 36mm In front of the plane of the filmstrip 10. A fllmgate 104 in the camera 
14 permits light from the scene that has passed the lens 102, to reach the filmstrip 10, thus forming a latent image of 
the scene. Rails 103 as shown In the side view, provide a robust surface against which the filmstrip 10 can slip as the 
filmstrip 10 advances. Light emitting diodes (LEDs) 101 are recessed Into the camera body. Each LED 101 may or 
may not be energized while the filmstrip 10 is stationary and being encoded with the desired infomriatlon. Individual 
55 bores in the camera's body carry light to the film. Alternate designs for delivering light to the film are well-known to 
those skilled In camera design. The light may be generated remotely and transported to the film by optbal elements. 
The light may be delivered to the front or to the back surface of the film. The light source may be modulated directly 
or a discrete light modulator such as an LCD may be used. 
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[0031] Fig. 5 shows a side view of the camera 14. A lens 102 is placed roughly 36mm rn front of the plane of filmstrip 
10. Rails 103 as shown In the side view, provide a robust surface against which the filmstrip 1 0 can slip as the filmstrip 
10 advances. A filmgate 104 in the camera 14 permits light from the scene that has passed the lens 102 to reach the 
filmstrip 10, thus forming a latent jmage of the scene. 

5 [0032] If the camera has a motor drive, a 1-dimensional bar code 20b can be implemented by the camera's 14 
mechanical configuration shown in Fig. 4b. Fig. 6 shows a rear view of the camera 14 against whic/i the filmstrip 10 
lies upon. A miniature optical projection assembly 108 is placed into a bore in the camera body. The assembly 108 
consists of an LED 101 that illuminates an aperture 106, and a lens 107 (shown, in Fig. 7) that focuses the aperture 
onto the filmstrip 10! This provides a nan^ow line of light on the filmstrip-IO. The LED is modulated In time to create the 

10 bar code 20b as the filmstrip 10 is moved to the next exposure position by the motor drive of the camera. Alternate 
designs for delivering light to the filmstrip 10 are well-known to those skilled in camera design. The light may be trans- 
ported to the filmstrip 10 by fiber optical elements or a small light source may be placed In direct contact with the 
filmstrip 10. The light may be delivered to the front or back surface of the filmstrip 10. Fig. 7 shows a side view of 
camera 14. Additional features include an aperture 106. and a lens 107 that focuses the aperture onto the filmstrip 10. 

15 [0033] A more sophisticated camera 14 could write one or more, 2-dimensional bar code blocks 20c by the mechan- 
ical configuration of camera 14 shown in Fig. 8. Fig. 8 shows a rear view of the camera 14, against which the filmstrip 
10 lies upon. Fig. 9 shows a side view of the camera 14. A lens 102 in Figs. 8 and 9 Is placed roughly 36mm In front 
of the plane of the filmstrip 10. A filmgate 104 In the camera 14 pemnits light from the scene, that has passed the lens 
102, to reach the filmstrip 10, thus forming a latent Image of the scene. Rails 103 that fomri a part of the filmgate 104 

20 as shown in the rear and side views, provide a robust surface against which the filmstrip 1 0 can slip as the filmstrip 10 
advances. 

[0034] Referring to Fig. 9, an Integral thin film technology liquid crystal display (TFT LCD) assembly 109 is placed 
close to the film. It is illuminated by a series of LEDs 1 01 that are recessed in the camera body, along the length of the 
LED assembly. During a period when the film is stationary, the signals are delivered to the LED array to energize 
25 selected transistor, rendering some pixels transparent and others opaque. All of the LEDs are subsequently energized 
. for a short period of time. 

[0035] In alt of the mechanical configurations noted above, the optical marks are written inside the safe frame area 
of each frame. In the embodiments shown in Figs. 4 through 9, the marks are written parallel to the long dimension of 
the film frame. And yet another embodiment of Figs. 4, 5, 8 and 9 can exclusively write the optical marks parallel to 
30 the short dimension of the film frame, or write optical marks of this sort in conjunction with the written marks that are 
parallel to the long dimension of the film frame. 

[0036] Referring to Fig. 1 , according to a preferred embodiment of the Invention, at the beginning and end of the 
filmstrip 10, one safe frame area 13 Is used to record a distinctive pattern 24 that can be detected in a photofinlshing 
operation to indicate that metadata is recorded on the film. In the case of a Single-Use camera which is loaded with 
35 film at the factory, this pattern 24 would be created by projection of light through a fixed mask or an LED-modulator 
onto the film. In the case of reloadable cameras, the pattern-generating means used for each frame would also be 
used to create this distinctive pattern. 

[0037] The apparatus for applying the metadata to the safe frame area of the film can also be incorporated In a single 
use camera, for example by suitably modifying the optical data recording circuit described In US Patent 5,870,639, 

40 referenced above, to pertonn the data exposure in safe frame area of the film. 

[0038] Referring to Fig. 3. a system 42 for providing photofinlshing according to the present invention includes a 
scanner 26, an Interface 28, an image data manager 30, a printer 34, and a digital output device 36. In operation, the 
scanner 26 scans the image on the safe areas 12 of film 10 in the normal manner to produce a digital Image. The 
digital image data from the scanner 26 Is passed through a standard hardware interface, such as a small computer 

45 system interface (SCSI) to the image data manager 30. The image data manager 30 includes a plurality of software 
components that implement various data handling and Image processing functions. Among the software components 
implemented in image data manager 30 are an interface component 28, an Image analysis component 38 and an 
image processing component 40. The interface component 28 usually perfomris three functions. 
[0039] First, the interface component 28 delivers commands from the Image data manager 30 to the scanner 26 and 

50 transmits status from the scanner 26 to the image data manager 30. These commands and status information must 
be translated from a scanner-dependent format to a scanner-independent fomiat, as expected by the image data 
manager 30. 

[0040] Second, the Interface component 28 stores the stream of Image data sent by the scanner 26 to the image 
data manager 30 In a temporary buffer, so that all the data from a single strip of film can be presented to analysis and 
55 processing algorithms. 

[0041] Third, the interface component 28 refonnats the stream of image data sent by the scanner 26 to the Image 
data manager 30, providing the Image information in the sequence and organization expected by the analysts and 
processing algorithms. The design and operation of the interface software is generally peculiar to a specific scanner 



INSDOCID: <EP 1213909A2J_> 



EP 1 213 909 A2 



and must be modified, if tlie functionality of the scanner is changed. This is often made mOre difficult by the fact that 
* portions of the interface software are implemented as driver components, within the operating system of the image data 
manager 30. It.is well known that operating system software Is more challenging to modify than application software. 
[0042] The contents of the frame 12 are designed so that known frameline detection algorithms used to locate the 

5 edges of the frames 12 will function ordinarily. The digital image data from the scanner 26 is passed through interface 
28 to the Image data manager 30. The image data manager 30 includes a plurality of components that implement 
various image processing functions. Among the software components implemented in data manager 30 are an image 
analysis component 38 and an image processing component 40. The image analysis component 38 provides direction 
to th€f image processing component 40 for each frame. The resulting processed digital image Is sent to a printer 34 or 

10 a digital writing device, e.g.. CD writer or an interface to the digital output device 36. 

[0043] The image analysis component 38 provides direction to the image processing component 40 for each frame. 
The primary functions of the image analysis component 38 are to classify the recorded image into one of a known set 
of image classes, such as outdoor, indoor with flash, or indoor with available light; or to measure a specific analog 
characteristic of the image. This characteristic may deal with the mechanistic exposure to the film, such as the mean 

IS optical density of the frame, or it may deal with some characteristic of the recorded scene, such as the location of open 
space within the image. In modern digital photofinishing systems the image analysis component 38 includes a large 
number of sub-analyses that neither limit the implementation of this invention, nor need to be modified in the presence 
of this invention. These sub-analyses are applied to the portion of the frame produced by the scene, as Is explained 
below. 

20 [0044] One of the sub-analyses impacts the implementation of this Invention. There is no unambiguous means to 
determine the location of image frames in the 35mm photographic system. A specific analysis algorithm, known as a 
frame-line detector, averages the optical density across the film width and identifies a pattern of sudden transitions 
from low density to high density orf rom high density to low density. From this pattern, the frame locations are discovered. 
It is a purpose of this invention to introduce metadata without disturbing the existing algorithms. If the metadata is 

2s written parallel to the long dimension of the film frame, the average density across the film will not be substantially 
disturbed. If the metadata is written parallel to the short dimension of the film frame, it must be designed so that the 
frame-line detector will reliably place the metadata inside the frame area, rather than outside the frame area. The 
placement of the metadata can be accomplished by assuring that at least half the width of the film is exposed for all 
possible coded messages. 

30 [0045] The optically written metadata is detected and interpreted by one sub-analysis element in the image analysis 
component 38. The relationship between the overall logical flow of the image analysis component 38 and this invention 
is shown in Fig. 10 with a flowchart. This sub-analysis acts on the temporary buffer representing the image data from 
the film strip, formed by the Interface component .28 and the output of the frame-line detector, also operating in the 
image analysis component 38. 

35 [0046] In the first operation 510, the frame-line algorithm is executed to identify the positions of exposed frames on 
the film. The result is a set of image data locations that mark the beginning of each frame. Operation 520, analyzes 
the presence of meaningful optical metadata. This analyzing process of operation 520 is further detailed in Fig. 11 . 
[0047] If metadata is present as determined in operation 530 and the capability to use it is contained in this version 
of digital photofinishing software, the metadata flag is set to true in operation 540. Operation 550 receives an indication 

40 from the metadata flag in operation 530 that no metadata is present and subsequently sets a metadata flag to a false 
state. Whereupon, operation 560 will point to the first frame, causing a metadata flag 570 to be analyzed further. If 
metadata flag 570 has a true state, then operation 580 must read the metadata in the first frame. Otherwise, operation 
590 will execute all the other sub-analyses that operate on the portion of the frame from the control table. A modification 
of the analyses based on the metadata is also performed. A recurring question is asked in operation 595: "are there 

45 more frames?" If the answer to operation 595 is affirmative, the metadata flag 570 must be analyzed for its current 
state again. Otherwise, the entire process ends at operation 597. 

[0048] The flowchart in Fig. 11 describes a software process for determining the presence of meaningful optical 
metadata that the process described in the flowchart of Fig. 10 will rely upon. Referring to Fig. 11 , the first operating 
step 61 0 examines a first frame for distinctive patterns. An initial analyzing step 620 detemrilnes whether metadata Is 
so present. No presence of metadata will cause operation 630 to examine the last frame for a distinctive pattern. A second 
analyzing step 640 also determines whether metadata is present. 

[0049] In the first analyzing step 620 an affirmative presence of metadata causes operation 650 to search a control 
table for metadata identification, otherwise described as ID. Operation 650 also stores control parameters for subse- 
quent usage. Operation 660 detemnlnes whether the metadata ID is known. Should the metadata ID be known, oper- 
55 atlon 670 causes a return to the process in flowchart with a result that metadata had been found, in contrast, if the 
metadata ID is unknown, operation 680 causes a return to the process in flowchart with a result that metadata had not 
been found. It should be noted that a negative presence of metadata as determined by operation 640 will also cause 
operation 680 to return to the process in flowchart with a result that metadata had not been found. 
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[0050] As previously described. Fig. 11 shows a method for determining whether a film strip contains meaninoful 
optical metadata. The process depends on the use of the first or last frame of the film to store distinctive patterns of 
density, and for identifying the type of metadata stored, if present. A distinctive pattern is defined as a very coarse 
pattern of high and low density areas which is unlikely to be created by any photographed scenes 
[0051] The first step 610 of the process in Fig. 1 1 is to examine the first frame discovered on the fil^i for a distinctive 
pattern. This specific process is further described below with reference to Fig. 12. if the pattern is not found the last 
frame is examined for the distinctive pattern once again using the process of Fig. 12; If no pattem Is found 'in either 
frame the nriain software logic flow, as described in Fig. 1 0. is inf omied that no metadata can be read from this particular 
film strip. If the distinctive pattem is found in either the first or last frame, the metadata ID is decoded from the pattem 
Those^skilled in the art will recognize that there are many ways to encode a number in binary patterns and that re- 
dundant codes can be used to reduce the probability of misinterpretation. 

(0052] The metadata ID found in the process shown in the flowchart of Fig. 1 1 is used to search a control table an 
example of which is disclosed below. 
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[0053] The control table includes six columns: a column for metadata ID designation; a column for metadata regions- 
a column for scene image regions; a column for metadata formats; a column for human readable tags; and the last 
column for feature set indexes. If the metadata ID is unknown, the main logical flow Is infomied that no metadata can 
be read from this film strip. If the metadata ID is present, the control parameters in the table are returned for further 
use. Their application is explained above. As noted in Fig. 11 . if the metadata is recognized, the control table exemplified 
above provides information about reading and using the metadata. The metadata ID is read from the firet frame or last 
frame via the process in Fig. 11 . The metadata ID is an identification number agreed on by the camera manufacturer 
and th^, digital photofinishing manufacturer. These manufacturers also share in the creation of the information recorded 

^® metadata region describes the portion of the usable film frame that contains metadata 
[0054] For example, where ID=1 in the control table, the metadata is stored across the bottom of the film frame 
parallel to it is long side. The scene image region describes the portion of the usable film frame, recorded on the filrri 
through the lens. Metadata format is an index to the storage format. For example, the code 1 might indfcate a specifk: 
dot code 20a; code 2, a speciffc one-dimensional barcode 20b; code 3. a specific two dimensional barcode 20c as 
desonbed earlier in Fig. 2. The human-readable tag and the feature set index are both indicators of the feature or 
feature set implemented by this metadata. The first indicator is used to track features by the system designers The 
second indicator is used in the software of the image analysis component 28 of Fig. 3 to access a software block that 
irnplements the feature. In this example, the control table describes for each metadata ID the physical region on the 
film where metadata should be found, the.physical region on the film where the scene image should be found and an 
index pointer to the metadata fonmat. «, ai lu oi i 

[0055] At this juncture, the image data representing each frame of the film is examined, beginning with the fii^t frame 
If the metadata flag is tme. the metadata is read from the frame. For a dot-code, the location of the dots on the frame 
IS stored m the software and linked to the metadata identification number read above. Each dot is recognized as beina 
a 0. If the pixel value at the known location is less than the mean pixel value in the dot-code region- othenvise the 
pixel value is recognized as being a 1. if the pixel value at the known location is greater than the mean pixel value in 
the dot-code region. Software for reading one-dimensional and two-dimensional barcodes is specific to the barcode 
used and is well known in the art. Note that the type and location of the metadata is known from the control table At 
this point all the rest of the sub-analyses contained in image analysis component 28 are executed. Only the portion of 
the frame identified by the control table as image is submitted to these sub-analyses. Some of these sub-analyses 
may be modified by metadata values read from this frame. These situations are identified based on the control table 
entry "feature set index." If there are more frames to be analyzed, this process repeats in a loop. Examination for a 
distinctive pattern, either found in the first or last frame of the film strip, is thus accomplished. This distinctive pattem 
provides a key to the fomiat and meaning of the metadata stored in each frame. The digital photofinishing system is 
capable of interpreting and acting on a specific instance of metadata only if its image analysis component 38 has been 
programmed to respond to Its specific key. ^ r 
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[0056] The collection of results 37 from these analyses Is sent toihe image processing component 40. The image 
processing component 40 perfoi-ms two key functions. First, It implements a set of rules that prescribe the Image 
processing steps required for a given combination of image analyses. Second, it executes a specific sequence of image 
processing steps on each frame of the image data provided by the interface component. 

5 [0057] For example, based bn fhe results of the analyjses, the image data corresponding to the third frame on the 
film strip might be passed through a specific three-dimensional lookup table and given a specific degree of image 
sharpening. The resultjng'processed digital image is sent to a printer 34 or a digital writing device, e.g., CD writer or 
an interface to the Internet 36. If the metadata extracted from the image by the irnage analysis component 38 is used 
by the Image data manager 30 for annotation rather than image modification, It is written on the back of the print by 

10 printer 34, or into the header of the digital file by digital output device 36, as Indicated by transfer path 32. 

[0058] Fig. 12 shows a method for implementing steps 610 and 630 of Fig. 11 in more detail. The method has been 
written to work with the distinctive pattem shown later in Fig. 13. The distinctive pattern has been co-designed with the 
method for detecting It. so as to make the method both simple and reliable. The input to the process diagrammed In 
Fig. 12 is the image data for the first or last film frame. The first operation 810 in the process is to measure the average 

15 pixel value in each of eight regions, each region running from the top to the bottom of the frame and together covering 
the entire frame. Next, an average of the total average pixel values 820 is conducted to find the mean value in the 
frame. This mean value is used to determine In operation 830 if. on average, each of the eight strip-averages Is above 
or below the mean. This produces from the image content eight bits, which can be joined in a fixed order to create an 
eight-bit number, ranging in valiJe from 0 to 255. The value generated In this process by the distinctive pattem in Fig. 

20 13 will not always be the same, due to shifts in the position of the pattem across the frame and due to the image 
content, but only a small subset of the numbers can be generated. A lookup operation based on the eight-bit number 
is used to identify a first aspect of the distinctive pattem, rejecting most scenes and all frames of unifonm value. This 
same lookup operation, part of 830, provides a set of image coordinates providing a means to further identify the 
distinctive pattern. 

25 [0059] If the lookup table entry is zero in inquiry operation 840, the distinctive pattem is likely not present, and op- 
eration 845 returns this information to the calling program of Fig.11. An example of a distinctive metadata pattern 900 
is shown in Fig. 14. In order to further determine the presence of the distinctive pattern 900, the lookup table contains 
the locations of the endpolnts of 6 lines, 910, 920, 930, 940, 950, and 960, as disclosed by Fig. 14. A reference line 
970 Is also shown In Fig. 1 4. If the pattem 900 is present, the mean value along lines 920 and 950 will be much higher 

30 than the mean values along lines 910, 930, 940, and 960. This set of lines has been co-designed with the distinctive 
pattem in Fig. 1 3. Operation 850 implements this test for a second aspect of the distinctive pattem. Again, this can be 
recognized via a lookup table entered from the 6 bits representing the intensities in the 6 lines. Operation 860 tests 
the entry in the lookup table, which will be a logical true if the pattem in the frame matches the distinctive pattern. Very 
few scenes contain a pattem of pixel values that match this distinctive pattern. If the pattern in the frame does not 

35 match the distinctive pattern, operation 845 returns this information to the calling program of Fig. 11. Hence. Fig. 14 
represents a tool for flowchart 800 in Fig. 12. It should be noted that Fig. 13 is merely one example of a possible 
distinctive pattern that could be implemented. 

[0060] If the pattem is recognized, operation 870 examines the image data in the frame at a plurality of locations 
905 noted in Fig. 13. These locations 905 contain high or low optical density as part of the distinctive pattern and serve 
40 to identify the specific content of the film strip. They are compared to the mean frame density to create a series of 
digital bits which are concatenated in a fixed sequence to fomi the metadata ID. This metadata ID Is returned to the 
calling program of Fig. 11 by operation 880. 

[0061] In another embodiment, where a standalone computer or retail photofinisher is used, the filmstrip 10 Is fed to 
a scanner 26, which is part of the digital photofinishing system 42. In contrast, for another embodiment, at a wholesale 

45 photofinisher, the filmstrip 10 shown In Fig. 3 is usually batched with many other rolls of filmstrip. to form a spool of 
film 50, joined together with splices 52, thus fomriing a different type of digital photofinishing system. Regardless which 
digital photofinishing system is used, because the pattern 20 is recorded on both ends of the filmstrip 1 0, either system 
can recognize the filmstrip 10 as special, and requiring special processing with the existing photofinishing hardware. 
A photofinisher need not create unique processing batches that contain film solely from these special metadata writing 

50 cameras. This feature of the present invention is critical because implementing a limited number of special cameras 
in an established photofinishing environment has to remain simple. A photofinisher need not modify her photofinishing 
hardware to process metadata laden film. 

[0062] The advantage of this invention may be appreciated by considering how few components of the photofinishing 
system 42 have to be modified to handle the metadata 20 encoded on the film. For example, the scanner 26, the 
55 Interface 28, and most of the image data manager 30, is indifferent to the presence of the metadata 20. A 35mm format 
photographic film may be used or another film format may be suitable. Only the image analysis component 38 must 
be modified to deal with the metadata 20 stored In this manner. The same system 42 can handle metadata 20 encoded 
as "dot-code" 20a, "1 -dimensional bar-code" 20b, or "2-dlmensional barcode" 20c or some intemilxed version of the 
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20 



three code types. • ' 

^ encoded in this manner is printed by a device not aware of the metadata, the resulting print 

will still be recognizable, although the metadata will be visible as well. In yet another embodiment, the coding system 
20c IS used, yielding metadata wherein the encoded information looks like a noisy gray border and results in the optical 
print being slightly less objectionable to a viewer. , 

Claims 

1 . A photofinishing method, comprising the steps of: 

a) exposing machine readable metadata and a scene image within an entire safe frame area on a filmstrip- 

b) processing the filmstrip to produce a visible image including the machine readable metadata and the scene 

image; 

c) scanning the safe frame area to produce a digital image; 

d) extracting the machine readable metadata from the digital image; 

e) extracting the scene image from the digital image; and 

f) processing the scene image according to the extracted machine readable jnetadata. 

2. The method claimed in claim 1 , further comprising the steps of: 

a) recording a distinctive pattern in the entire safe frame area at a beginning and end of the filmstrip to indicate 
that the filmstrip contains the machine readable metadata; and 

b) detecting the distinctive pattern on the filmstrip prior to extracting the machine readable metadata from the 
digital image. 

3. The method claimed in claim 1 , further comprising the steps of: 

refomnatting the processed scene image; and 
printing the processed scene image. 

4. The method claimed in claim 1, wherein the machine readable metadata and the scene image are exposed in 
separate portions of the safe frame area. 

5. The method claimed in claim 4, wherein the machine readable metadata is isolated from the scene imaqe bv a 
distinctive color pattern. ^ ^ 

6. The method claimed in claim 1 , further comprising the steps of: 

passing the filmstrip through hardware interfaces; 

b) passing the filmstrip through software interfaces; 

c) delivering the filmstrip to a computational engine; 

d) decoding the filmstrip; and 

e) modifying the scene image data to produce a digital image. 
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